Skip to main content

Reply to this post | See parent post | Go Back
View Post [edit]

Poster: brewster Date: Jan 29, 2008 12:45pm
Forum: web Subject: Re: The Internet Archive Censorship Scandal

There was nothing special happening before 9/11/2001-- the attempt is to crawl every site ever 2 months. sometimes things would be more frequent, but mostly 2 months.

with 9/11 events, the crawler team tried to archive things much more frequently. the news sites had trouble staying up, so that record is a bit spotty.

I hope this helps.